Auditory cortex conveys non-topographic sound localization signals to visual cortex.

Spatiotemporally congruent sensory stimuli are fused into a unified percept. The auditory cortex (AC) sends projections to the primary visual cortex (V1), which could provide signals for binding spatially corresponding audio-visual stimuli. However, whether AC inputs in V1 encode sound location remains unknown. Using two-photon axonal calcium imaging and a speaker array, we measured the auditory spatial information transmitted from AC to layer 1 of V1. AC conveys information about the location of ipsilateral and contralateral sound sources to V1. Sound location could be accurately decoded by sampling AC axons in V1, providing a substrate for making location-specific audiovisual associations. However, AC inputs were not retinotopically arranged in V1, and audio-visual modulations of V1 neurons did not depend on the spatial congruency of the sound and light stimuli. The non-topographic sound localization signals provided by AC might allow the association of specific audiovisual spatial patterns in V1 neurons.

Eye blinks as a visual processing stage.

Humans blink their eyes frequently during normal viewing, more often than it seems necessary for keeping the cornea well lubricated. Since the closure of the eyelid disrupts the image on the retina, eye blinks are commonly assumed to be detrimental to visual processing. However, blinks also provide luminance transients rich in spatial information to neural pathways highly sensitive to temporal changes. Here, we report that the luminance modulations from blinks enhance visual sensitivity. By coupling high-resolution eye tracking in human observers with modeling of blink transients and spectral analysis of visual input signals, we show that blinking increases the power of retinal stimulation and that this effect significantly enhances visibility despite the time lost in exposure to the external scene. We further show that, as predicted from the spectral content of input signals, this enhancement is selective for stimuli at low spatial frequencies and occurs irrespective of whether the luminance transients are actively generated or passively experienced. These findings indicate that, like eye movements, blinking acts as a computational component of a visual processing strategy that uses motor behavior to reformat spatial information into the temporal domain.

Effects of neural oscillation power and phase on discrimination performance in a visual tilt illusion.

Neural oscillations reflect fluctuations in the relative excitation/inhibition of neural systems1,2,3,4,5 and are theorized to play a critical role in canonical neural computations6,7,8,9 and cognitive processes.10,11,12,13,14 These theories have been supported by findings that detection of visual stimuli fluctuates with the phase of oscillations prior to stimulus onset.15,16,17,18,19,20,21,22,23 However, null results have emerged in studies seeking to demonstrate these effects in visual discrimination tasks,24,25,26,27 raising questions about the generalizability of these phenomena to wider neural processes. Recently, we suggested that methodological limitations may mask effects of phase in higher-level sensory processing.28 To test the generality of phasic influences on perception requires a task that involves stimulus discrimination while also depending on early sensory processing. Here, we examined the influence of oscillation phase on the visual tilt illusion, in which a center grating has its perceived orientation biased away from the orientation of a surround grating29 due to lateral inhibitory interactions in early visual processing.30,31,32 We presented center gratings at participants' subjective vertical angle and had participants report whether the grating appeared tilted clockwise or counterclockwise from vertical on each trial while measuring their brain activity with electroencephalography (EEG). In addition to effects of alpha power and aperiodic slope, we observed robust associations between orientation perception and alpha and theta phase, consistent with fluctuating illusion magnitude across the oscillatory cycle. These results confirm that oscillation phase affects the complex processing involved in stimulus discrimination, consistent with its purported role in canonical computations that underpin cognition.

An open platform for visual stimulation of insects.

To study how the nervous system processes visual information, experimenters must record neural activity while delivering visual stimuli in a controlled fashion. In animals with a nearly panoramic field of view, such as flies, precise stimulation of the entire visual field is challenging. We describe a projector-based device for stimulation of the insect visual system under a microscope. The device is based on a bowl-shaped screen that provides a wide and nearly distortion-free field of view. It is compact, cheap, easy to assemble, and easy to operate using the included open-source software for stimulus generation. We validate the virtual reality system technically and demonstrate its capabilities in a series of experiments at two levels: the cellular, by measuring the membrane potential responses of visual interneurons; and the organismal, by recording optomotor and fixation behavior of Drosophila melanogaster in tethered flight. Our experiments reveal the importance of stimulating the visual system of an insect with a wide field of view, and we provide a simple solution to do so.

Neural assemblies coordinated by cortical waves are associated with waking and hallucinatory brain states.

The relationship between sensory stimuli and perceptions is brain-state dependent: in wakefulness, suprathreshold stimuli evoke perceptions; under anesthesia, perceptions are abolished; and during dreaming and in dissociated states, percepts are internally generated. Here, we exploit this state dependence to identify brain activity associated with internally generated or stimulus-evoked perceptions. In awake mice, visual stimuli phase reset spontaneous cortical waves to elicit 3-6 Hz feedback traveling waves. These stimulus-evoked waves traverse the cortex and entrain visual and parietal neurons. Under anesthesia as well as during ketamine-induced dissociation, visual stimuli do not disrupt spontaneous waves. Uniquely, in the dissociated state, spontaneous waves traverse the cortex caudally and entrain visual and parietal neurons, akin to stimulus-evoked waves in wakefulness. Thus, coordinated neuronal assemblies orchestrated by traveling cortical waves emerge in states in which perception can manifest. The awake state is privileged in that this coordination is reliably elicited by external visual stimuli.

The time course of feature-selective attention inside and outside the focus of spatial attention.

Research on attentional selection of stimulus features has yielded seemingly contradictory results. On the one hand, many experiments in humans and animals have observed a "global" facilitation of attended features across the entire visual field, even when spatial attention is focused on a single location. On the other hand, several event-related potential studies in humans reported that attended features are enhanced at the attended location only. The present experiment demonstrates that these conflicting results can be explained by differences in the timing of attentional allocation inside and outside the spatial focus of attention. Participants attended to fields of either red or blue randomly moving dots on either the left or right side of fixation with the task of detecting brief coherent motion targets. Recordings of steady-state visual evoked potentials elicited by the flickering stimuli allowed concurrent measurement of the time course of feature-selective attention in visual cortex on both the attended and the unattended sides. The onset of feature-selective attentional modulation on the attended side occurred around 150 ms earlier than on the unattended side. This finding that feature-selective attention is not spatially global from the outset but extends to unattended locations after a temporal delay resolves previous contradictions between studies finding global versus hierarchical selection of features and provides insight into the fundamental relationship between feature-based and location-based (spatial) attention mechanisms.

Auditory and Visual Gratings Elicit Distinct Gamma Responses.

Sensory stimulation is often accompanied by fluctuations at high frequencies (>30 Hz) in brain signals. These could be "narrowband" oscillations in the gamma band (30-70 Hz) or nonoscillatory "broadband" high-gamma (70-150 Hz) activity. Narrowband gamma oscillations, which are induced by presenting some visual stimuli such as gratings and have been shown to weaken with healthy aging and the onset of Alzheimer's disease, hold promise as potential biomarkers. However, since delivering visual stimuli is cumbersome as it requires head stabilization for eye tracking, an equivalent auditory paradigm could be useful. Although simple auditory stimuli have been shown to produce high-gamma activity, whether specific auditory stimuli can also produce narrowband gamma oscillations is unknown. We tested whether auditory ripple stimuli, which are considered an analog to visual gratings, could elicit narrowband oscillations in auditory areas. We recorded 64-channel electroencephalogram from male and female (18 each) subjects while they either fixated on the monitor while passively viewing static visual gratings or listened to stationary and moving ripples, played using loudspeakers, with their eyes open or closed. We found that while visual gratings induced narrowband gamma oscillations with suppression in the alpha band (8-12 Hz), auditory ripples did not produce narrowband gamma but instead elicited very strong broadband high-gamma response and suppression in the beta band (14-26 Hz). Even though we used equivalent stimuli in both modalities, our findings indicate that the underlying neuronal circuitry may not share ubiquitous strategies for stimulus processing.

Perceptography unveils the causal contribution of inferior temporal cortex to visual perception.

Neurons in the inferotemporal (IT) cortex respond selectively to complex visual features, implying their role in object perception. However, perception is subjective and cannot be read out from neural responses; thus, bridging the causal gap between neural activity and perception demands independent characterization of perception. Historically, though, the complexity of the perceptual alterations induced by artificial stimulation of IT cortex has rendered them impossible to quantify. To address this old problem, we tasked male macaque monkeys to detect and report optical impulses delivered to their IT cortex. Combining machine learning with high-throughput behavioral optogenetics, we generated complex and highly specific images that were hard for the animal to distinguish from the state of being cortically stimulated. These images, named "perceptograms" for the first time, reveal and depict the contents of the complex hallucinatory percepts induced by local neural perturbation in IT cortex. Furthermore, we found that the nature and magnitude of these hallucinations highly depend on concurrent visual input, stimulation location, and intensity. Objective characterization of stimulation-induced perceptual events opens the door to developing a mechanistic theory of visual perception. Further, it enables us to make better visual prosthetic devices and gain a greater understanding of visual hallucinations in mental disorders.

Beyond visual integration: sensitivity of the temporal-parietal junction for objects, places, and faces.

One important role of the TPJ is the contribution to perception of the global gist in hierarchically organized stimuli where individual elements create a global visual percept. However, the link between clinical findings in simultanagnosia and neuroimaging in healthy subjects is missing for real-world global stimuli, like visual scenes. It is well-known that hierarchical, global stimuli activate TPJ regions and that simultanagnosia patients show deficits during the recognition of hierarchical stimuli and real-world visual scenes. However, the role of the TPJ in real-world scene processing is entirely unexplored. In the present study, we first localized TPJ regions significantly responding to the global gist of hierarchical stimuli and then investigated the responses to visual scenes, as well as single objects and faces as control stimuli. All three stimulus classes evoked significantly positive univariate responses in the previously localized TPJ regions. In a multivariate analysis, we were able to demonstrate that voxel patterns of the TPJ were classified significantly above chance level for all three stimulus classes. These results demonstrate a significant involvement of the TPJ in processing of complex visual stimuli that is not restricted to visual scenes and that the TPJ is sensitive to different classes of visual stimuli with a specific signature of neuronal activations.

Temporal windows of unconscious processing cannot easily be disrupted.

Conscious perception is preceded by long periods of unconscious processing. These periods are crucial for analyzing temporal information and for solving the many ill-posed problems of vision. An important question is what starts and ends these windows and how they may be interrupted. Most experimental paradigms do not offer the methodology required for such investigation. Here, we used the sequential metacontrast paradigm, in which two streams of lines, expanding from the center to the periphery, are presented, and participants are asked to attend to one of the motion streams. If several lines in the attended motion stream are offset, the offsets are known to integrate mandatorily and unconsciously, even if separated by up to 450 ms. Using this paradigm, we here found that external visual objects, such as an annulus, presented during the motion stream, do not disrupt mandatory temporal integration. Thus, if a window is started once, it appears to remain open even in the presence of disruptions that are known to interrupt visual processes normally. Further, we found that interrupting the motion stream with a gap disrupts temporal integration but does not terminate the overall unconscious processing window. Thus, while temporal integration is key to unconscious processing, not all stimuli in the same processing window are integrated together. These results strengthen the case for unconscious processing taking place in windows of sensemaking, during which temporal integration occurs in a flexible and perceptually meaningful manner.

Common structure of saccades and microsaccades in visual perception.

We obtain large amounts of external information through our eyes, a process often considered analogous to picture mapping onto a camera lens. However, our eyes are never as still as a camera lens, with saccades occurring between fixations and microsaccades occurring within a fixation. Although saccades are agreed to be functional for information sampling in visual perception, it remains unknown if microsaccades have a similar function when eye movement is restricted. Here, we demonstrated that saccades and microsaccades share common spatiotemporal structures in viewing visual objects. Twenty-seven adults viewed faces and houses in free-viewing and fixation-controlled conditions. Both saccades and microsaccades showed distinctive spatiotemporal patterns between face and house viewing that could be discriminated by pattern classifications. The classifications based on saccades and microsaccades could also be mutually generalized. Importantly, individuals who showed more distinctive saccadic patterns between faces and houses also showed more distinctive microsaccadic patterns. Moreover, saccades and microsaccades showed a higher structure similarity for face viewing than house viewing and a common orienting preference for the eye region over the mouth region. These findings suggested a common oculomotor program that is used to optimize information sampling during visual object perception.

Effects of expectation on face perception and its association with expertise.

Perceptual decisions are derived from the combination of priors and sensorial input. While priors are broadly understood to reflect experience/expertise developed over one's lifetime, the role of perceptual expertise at the individual level has seldom been directly explored. Here, we manipulate probabilistic information associated with a high and low expertise category (faces and cars respectively), while assessing individual level of expertise with each category. 67 participants learned the probabilistic association between a color cue and each target category (face/car) in a behavioural categorization task. Neural activity (EEG) was then recorded in a similar paradigm in the same participants featuring the previously learned contingencies without the explicit task. Behaviourally, perception of the higher expertise category (faces) was modulated by expectation. Specifically, we observed facilitatory and interference effects when targets were correctly or incorrectly expected, which were also associated with independently measured individual levels of face expertise. Multivariate pattern analysis of the EEG signal revealed clear effects of expectation from 100 ms post stimulus, with significant decoding of the neural response to expected vs. not stimuli, when viewing identical images. Latency of peak decoding when participants saw faces was directly associated with individual level facilitation effects in the behavioural task. The current results not only provide time sensitive evidence of expectation effects on early perception but highlight the role of higher-level expertise on forming priors.

The other-race effect of pupil contagion in infancy.

Pupil contagion refers to the observer's pupil-diameter changes in response to changes in the pupil diameter of others. Recent studies on the other-race effect on pupil contagion have mainly focused on using eye region images as stimuli, revealing the effect in adults but not in infants. To address this research gap, the current study used whole-face images as stimuli to assess the pupil-diameter response of 5-6-month-old and 7-8-month-old infants to changes in the pupil-diameter of both upright and inverted unfamiliar-race faces. The study initially hypothesized that there would be no pupil contagion in either upright or inverted unfamiliar-race faces, based on our previous finding of pupil contagion occurring only in familiar-race faces among 5-6-month-old infants. Notably, the current results indicated that 5-6-month-old infants exhibited pupil contagion in both upright and inverted unfamiliar-race faces, while 7-8-month-old infants showed this effect only in upright unfamiliar-race faces. These results demonstrate that the face inversion effect of pupil contagion does not occur in 5-6-month-old infants, thereby suggesting the presence of the other-race effect in pupil contagion among this age group. Overall, this study provides the first evidence of the other-race effect on infants' pupil contagion using face stimuli.

Functional OCT reveals anisotropic changes of retinal flicker-evoked vasodilation.

The purpose of this study is to verify the effect of anisotropic property of retinal biomechanics on vasodilation measurement. A custom-built optical coherence tomography (OCT) was used for time-lapse imaging of flicker stimulation-evoked vessel lumen changes in mouse retinas. A comparative analysis revealed significantly larger (18.21%) lumen dilation in the axial direction compared to the lateral (10.77%) direction. The axial lumen dilation predominantly resulted from the top vessel wall movement toward the vitreous direction, whereas the bottom vessel wall remained stable. This observation indicates that the traditional vasodilation measurement in the lateral direction may result in an underestimated value.

Willed Attentional Selection of Visual Features: An EEG Study.

Visual selective attention studies generally tend to apply cuing paradigms to instructively direct observers' attention to certain locations, features or objects. However, in real situations, attention in humans often flows spontaneously without any specific instructions. Recently, a concept named "willed attention" was raised in visuospatial attention, in which participants are free to make volitional attention decisions. Several ERP components during willed attention were found, along with a perspective that ongoing alpha activity may bias the subsequent attentional choice. However, it remains unclear whether similar neural mechanisms exist in feature- or object-based willed attention. Here, we included choice cues and instruct cues in a feature-based selective attention paradigm, allowing participants to freely choose or to be instructed to attend a color for the subsequent target detection task. Pre-cue ongoing alpha oscillations, cue-evoked potentials and target-related steady-state visual evoked potentials (SSVEPs) were simultaneously measured as markers of attentional processing. As expected, SSVEP responses were similarly modulated by attention between choice and instruct cue trials. Similar to the case of spatial attention, a willed-attention component (Willed Attention Component, WAC) was isolated during the cue-related choice period by comparing choice and instruct cues. However, pre-cue ongoing alpha oscillations did not predict the color choice (yellow vs blue), as indicated by the chance level decoding accuracy (50%). Overall, our results revealed both similarities and differences between spatial and feature-based willed attention, and thus extended the understanding toward the neural mechanisms of volitional attention.

A Time-Local Weighted Transformation Recognition Framework for Steady State Visual Evoked Potentials Based Brain-Computer Interfaces.

Canonical correlation analysis (CCA), Multivariate synchronization index (MSI), and their extended methods have been widely used for target recognition in Brain-computer interfaces (BCIs) based on Steady State Visual Evoked Potentials (SSVEP), and covariance calculation is an important process for these algorithms. Some studies have proved that embedding time-local information into the covariance can optimize the recognition effect of the above algorithms. However, the optimization effect can only be observed from the recognition results and the improvement principle of time-local information cannot be explained. Therefore, we propose a time-local weighted transformation (TT) recognition framework that directly embeds the time-local information into the electroencephalography signal through weighted transformation. The influence mechanism of time-local information on the SSVEP signal can then be observed in the frequency domain. Low-frequency noise is suppressed on the premise of sacrificing part of the SSVEP fundamental frequency energy, the harmonic energy of SSVEP is enhanced at the cost of introducing a small amount of high-frequency noise. The experimental results show that the TT recognition framework can significantly improve the recognition ability of the algorithms and the separability of extracted features. Its enhancement effect is significantly better than the traditional time-local covariance extraction method, which has enormous application potential.

Multi-Stimulus Least-Squares Transformation With Online Adaptation Scheme to Reduce Calibration Effort for SSVEP-Based BCIs.

Steady-state visual evoked potential (SSVEP), one of the most popular electroencephalography (EEG)-based brain-computer interface (BCI) paradigms, can achieve high performance using calibration-based recognition algorithms. As calibration-based recognition algorithms are time-consuming to collect calibration data, the least-squares transformation (LST) has been used to reduce the calibration effort for SSVEP-based BCI. However, the transformation matrices constructed by current LST methods are not precise enough, resulting in large differences between the transformed data and the real data of the target subject. This ultimately leads to the constructed spatial filters and reference templates not being effective enough. To address these issues, this paper proposes multi-stimulus LST with online adaptation scheme (ms-LST-OA). METHODS: The proposed ms-LST-OA consists of two parts. Firstly, to improve the precision of the transformation matrices, we propose the multi-stimulus LST (ms-LST) using cross-stimulus learning scheme as the cross-subject data transformation method. The ms-LST uses the data from neighboring stimuli to construct a higher precision transformation matrix for each stimulus to reduce the differences between transformed data and real data. Secondly, to further optimize the constructed spatial filters and reference templates, we use an online adaptation scheme to learn more features of the EEG signals of the target subject through an iterative process trial-by-trial. RESULTS: ms-LST-OA performance was measured for three datasets (Benchmark Dataset, BETA Dataset, and UCSD Dataset). Using few calibration data, the ITR of ms-LST-OA achieved 210.01±10.10 bits/min, 172.31±7.26 bits/min, and 139.04±14.90 bits/min for all three datasets, respectively. CONCLUSION: Using ms-LST-OA can reduce calibration effort for SSVEP-based BCIs.

Short communication: Binocular rivalry dynamics during locomotion.

Locomotion has been shown to impact aspects of visual processing in both humans and animal models. In the current study, we assess the impact of locomotion on the dynamics of binocular rivalry. We presented orthogonal gratings, one contrast-modulating at 0.8 Hz (matching average step frequency) and the other at 3.2 Hz, to participants using a virtual reality headset. We compared two conditions: stationary and walking. We continuously monitored participants' foot position using tracking devices to measure the step cycle. During the walking condition, participants viewed the rivaling gratings for 60-second trials while walking on a circular path in a virtual reality environment. During the stationary condition, observers viewed the same stimuli and environment while standing still. The task was to continuously indicate the dominant percept via button press using handheld controllers. We found no significant differences between walking and standing for normalized dominance duration distributions, mean normalized dominance distributions, mean alternation rates, or mean fitted frequencies. Although our findings do not align with prior research highlighting distinctions in normalized dominance distributions between walking and standing, our study contributes unique evidence indicating that alternation rates vary across the step cycle. Specifically, we observed that the number of alternations is at its lowest during toe-off phases and reaches its peak at heel strike. This novel insight enhances our understanding of the dynamic nature of alternation patterns throughout the step cycle.

Effectiveness of binaural beat music combined with rhythmical photic stimulation on older people with depressive symptoms in long-term care institution: a quasi-experimental pilot study.

BACKGROUND: Many older adults residing in long-term care often face issues like poor sleep, reduced vitality, and depression. Non-pharmacological approaches, specifically Binaural Beat Music (BBM) and Rhythmic Photic Stimulation (RPS), may alleviate these symptoms, yet their efficacy in this demographic has not been extensively explored. AIMS: This study investigated the effects of combined BBM and RPS interventions on sleep quality, vitality, and depression among older residents with depressive symptoms in long-term care facilities. METHODS: Using a quasi-experimental design, a total of 88 older adults with depressive symptoms from Taiwanese daytime care centers were divided into the BBM with RPS, and Sham groups (44 each). They underwent 20-minute daily sessions of their assigned treatment for two weeks. The BBM with RPS group listened to 10 Hz binaural beat music with 10 Hz photic stimulation glasses, and the Sham group received non-stimulating music and glasses. RESULTS: After the intervention, participants in the BBM with RPS groups showed significant improvements in vitality and depressive mood, with a notable increase in sympathetic nervous system activity. Conversely, the Sham group exhibited significant deterioration in vitality and mental health, with a significant increase in parasympathetic activity. Additionally, compared with the Sham group, the BBM and RPS groups showed significant improvements in vitality, mental health, and depression, with a significant increase in sympathetic nervous activity. CONCLUSION: The two-week intervention suggests that the combination of BBM and RPS, as a non-invasive intervention, can potentially improve vitality, mental health, and depressive mood among older adults in long-term care institutions.

Statistical Evaluation of ERG Responses: A New Method to Validate Cycle-by-Cycle Recordings in Advanced Retinal Degenerations.

Purpose: To describe and evaluate a novel method to determine the validity of measurements made using cycle-by-cycle (CxC) recording techniques in patients with advanced retinal degenerations (RD) having low-amplitude flicker electroretinogram (ERG) responses. Methods: The method extends the original CxC recording algorithm introduced by Sieving et al., retaining the original recording setup and the preliminary analysis of raw data. Novel features include extended use of spectrum analysis, reduction of errors due to known sources, and a comprehensive statistical assessment using three different tests. The method was applied to ERG recordings from seven patients with RD and two patients with CNGB3 achromatopsia. Results: The method was implemented as a Windows application to processes raw data obtained from a commercial ERG system, and it features a computational toolkit for statistical assessment of ERG recordings with amplitudes as low as 1 µV, commonly found in advanced RD patients. When recorded using conditions specific for eliciting cone responses, none of the CNGB3 patients had a CxC validated response, indicating that no signal artifacts were present with our recording conditions. A comparison of the presented method with conventional 30 Hz ERG was performed. Bland-Altman plots indicated good agreement (mean difference, -0.045 µV; limits of agreement, 0.193 to -0.282 µV) between the resulting amplitudes. Within-session test-retest variability was 15%, comparing favorably to the variability of standard ERG amplitudes. Conclusions: This novel method extracts highly reliable clinical recordings of low-amplitude flicker ERGs and effectively detects artifactual responses. It has potential value both as a cone outcome variable and planning tool in clinical trials on natural history and treatment of advanced RDs.